Speech messages, modulations, and motions

Temporal modulations in speech and music.

Speech and music have structured rhythms. Here we discuss a major acoustic correlate of spoken and musical rhythms, the slow (0.25-32Hz) temporal modulations in sound intensity and compare the modulation properties of speech and music. We analyze these modulations using over 25h of speech and over 39h of recordings of Western music. We show that the speech modulation spectrum is highly consiste...

Speech recognition with amplitude and frequency modulations.

Amplitude modulation (AM) and frequency modulation (FM) are commonly used in communication, but their relative contributions to speech recognition have not been fully explored. To bridge this gap, we derived slowly varying AM and FM from speech sounds and conducted listening tests using stimuli with different modulations in normal-hearing and cochlear-implant subjects. We found that although AM...

Spectro-temporal Modulations for Robust Speech Emotion Recognition Spectro-temporal Modulations for Robust Speech Emotion Recognition

Speech - Nonspeech discrimination based on speech-relevant spectrogram modulations

In this work, we adopt an information theoretic approach the Information Bottleneck method to extract the relevant modulation frequencies across both dimensions of a spectrogram, for speech / non-speech discrimination (music, animal vocalizations, environmental noises). A compact representation is built for each sound ensemble, consisting of the maximally informative features. We demonstrate th...

Auditory-Visual Speech Recognition with Amplitude and Frequency Modulations

A recent study by Zeng et al (2005) [PNAS, 102, 2293-2298] demonstrated the importance of FM cues for auditory speech identification in a competing noise environment. The current speech identification study investigated this finding for both an Auditory Only (AO) and an Auditory-Visual (AV) speech in noise identification task. The results demonstrated an FM advantage (compared to AM only) for b...